EAN: Event Adaptive Network for Enhanced Action Recognition

Efficiently modeling spatial–temporal information in videos is crucial for action recognition. To achieve this goal, state-of-the-art methods typically employ the convolution operator and dense interaction modules such as non-local blocks. However, these cannot accurately fit diverse events videos. On one hand, adopted convolutions are with fixed scales, thus struggling of various scales. other paradigm only achieves sub-optimal performance action-irrelevant parts bring additional noises final prediction. In paper, we propose a unified recognition framework to investigate dynamic nature video content by introducing following designs. First, when extracting local cues, generate kernels dynamic-scale adaptively events. Second, aggregate cues into global representation, mine interactions among few selected foreground objects Transformer, which yields sparse paradigm. We call proposed Event Adaptive Network because both key designs adaptive input content. exploit short-term motions within segments, novel efficient Latent Motion Code module, further improving framework. Extensive experiments on several large-scale datasets, e.g., Something-to-Something V1 &V2, Kinetics, Diving48, verify that our models or competitive performances at low FLOPs. Codes available at: https://github.com/tianyuan168326/EAN-Pytorch .